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Prefatory Note 



These papers were presented m a symposium entitled, Project 
REALISTIC; Evaluation and Modification of REAding, Listening, and 
ArithmeTIC Requirements in Military Occupations Having Civihasn 
Counterparts,” at the Western Psychological Association Annu^ meeUng, 
April 1970. The papers present findings from Work Unit REALISllC, 
Determination of Reading, Listening, and Arithmetic Skills Reqmred far 
Major Military Occupational Specialties, a program of rese^ch being 
performed by the Human Resources Research Organization, Division No. 

3, at Presidio of Monterey, California. 

Following the presentation of these papers, they were discussed by 
■ Dr Donald Ross Green of the California Test Bureau and Mr. J. 
Edmond Phillips of the Department of Human Resources Deve^pment, 
Occupational Analysis Field Center at Los Angeles, California, ^aarm^ 
of the symposium was Dr. Howard H. McFann, Director, HumRRO 

Division No. 3. 
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OVERVIEW OF PROJECT REALISTIC 



Thomas G. Sticht 



In 1966, the armed services initiated Project 100,000 in which the armed services 
joined with other Federal agencies in the war against p'-verty by admitting men who were 
previously disqualified for service because of low mental aptitude or, in fewer cases, 

deficient health standards. . 

Among the factors leading to the implementation of Project 100,000 was a growmg 

conviction that “ . . . the prime reason many men ‘fail’ the aptitude tests given at the 
time of induction is simply that these tests are geared to the psychology of tradition^, 
formal, classroom, teacher-paced instruction.^” And that “ . . . these tests inevstebly 
reflect the cultural value-systems and verbal patterns of affluent Amencan Society. That 
is why so many young men from poverty backgrounds do poorly in the test. It is not 
because they do not possess basic— and perhaps even brilliant— intelligence; but, simply 
because their cultural environment is so radically different from that assumed Dy the tes 
designers.’” For these reasons, it was felt that many potentially trainable men were 
“screened out” of the armed services, and hence being denied the education, job skill, 
and health benefits of the services, for reasons of past failure, rather than being screened 
in” for future development and accomplishment. 

A Question of Literacy 

With the implementation of Project 100,000 it was anticipated that there would be 
an influx of many men of marginal, literacy into the services. This was confirm^ "V 
statistics in 1968 which indicated that 31% of a sample of 46,000 Project 100,. 00 .. 

read at or below the fourth grade level of ability. Some 68% of these low aptitude men 

read at or below the sixth grade level of abUity. 

Because of the low reading skills of many of the Project 100,000 men, there w 
concern among manpower specialists that many “new standards” men might be assigned 
to jobs wherein the demands for reading skUls might far exceed the abUity of the men, 
and possibly lead to job failure. To reduce this possibility, information about the literacy 
demands of military jobs was needed. Such information is valuable for decision making 
regarding classification of men in job areas, for deciding on niinimal levels reqUi. e in 
adult basic education to render a person qualified for a given job, and for determining 
whether the literacy requirements of jobs might be altered through the redesipi of 
reading materials, provision of audio materials, and so forth. Such redesign of matenals 
could possibly reduce entrance requirements in literacy skills so that persons deficient in 
such skills could be effectively utilizea, perhaps while receiving training to improve their 
literacy skills, and thus prepare themselves for higher level job positions. 




Project REALISTIC 

To obtain information concerning the literacy demands of military jobs. Work Unit 
realistic was initiated in 1968. REALISTIC is an acronym based upon the three 
literacy skill areas studied-REAding, UStening, and ArithmeTIC. The general objectives 



* Address by Robert S. McNamara, former Secretary of Defense, to the Veterans of Foreign Wars, 
New York, 22 August 1966, 



6 



1 



of Work Unit REALISTIC, which is currently in progress, are to (a) provide information 
concerning the demands for reading, listening, and arithmetic skills in several major - 
military occupational specialties (MOSs), and (b) provide iniormation and suggestions for 
reduci^ discrepancies between personnel literacy skill levels and the literacy skill levels 

required objective, vre are examining relationships between the reading, 

listening and arithmetic skills of job incumbents and their performance on four different 
indices of job proficiency. As Figure 1 indicates, the proficiency indices we are using 
include proficiency on job-related reading tasks, proficiency op hands-on job sample 
performance tests, proficiency on job knowledge (paper-and-pencil) tests, and proficiency 

“ literacy aidlls and proflcency on the Job ^rfornran^ 

test, knowledge test, and supervisor’s ratings are being performed under REALISTIC . 
Data from this effort follow in the first paper in this series, by Dr. . 

“Psychometric Determination of Relationships Between Literacy Skills and Job 

Perfonnmicenshi literacy skills and performance on job-related reading tasks 

are being explored under REALISTIC II. Activities under REALISTIC H have ^^en (or 
are being) accomplished to identify job-related reading tasks. These 

discussed in the second paper by Dr. Kern on Reading * ‘ rieVks and 

Readership: Identifying Job-Related Reading Tasks Performed by Cooks, Clerks, 

MechanicSthi^^ sub-unit of REALISTIC attempts to identify information and to make 
suggestions useful for reducing gaps that may exist between an individual s literacy s^ll 
lev?i and the demands of the job for these s^jlls. This sub-unit has involved some 
experimental work, primarily in regard to listening, but is otherwise mostly a literature 
suLy type of effort As indicated in Figure 1, we have considered reducing discrepancies 
beZeiX skUls people have and the sidlla needed tor the i-b by e.ther modifying th 
nerson through literacy training, or by modifying the job demands. In the latter case w 
tave considered the substitution of listening tor reading demands and 

and training aids to make them less demanding ot literacy skills. These topics are 
discussed at greater length in the last paper “Reducing Disc.repanc.es Between Literacy 

Skill Levels of Personnel and Literacy Demands of Jobs. , . • i nf 

- The emphasis of this research upon identifying genuine educational reqmrements of 
jobs as a step towards redressing the social ills resulting from inflated ^redentia 
requirements, represents a major shift in the applications of psychometric and behavioral 
analysis techniques for matching people and jobs. 
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Organizational Chart for Work Unit REALISTIC 
Literacy Skills in Relation To ^ 



Job-Related 
Reading Tasks 

/ V 
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Do Not 
Use 

Manuals 



— I — 
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Reading 
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For Reading 
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III 



Reading 
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Rate 

Controlled 
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In Sub'unit ReX^ISTIC I, literacy skills are related to performance on job performance, knowledge, 
and supervisor rat ings . IjrR E A LIST 1C II, literacy skills are related to job-related reading tasks. In 
REALISTIC III, methods of matching literacy skills of men to literacy demands of jobs are considered. 



Figure 1 
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PSYCHOMETRIC DETERMINATION OF RELATIONSHIPS AMONG 
LITERACY SKILLS AND JOB PERFORMANCE 

John S. Caylor 
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One of the primary objectives of Work Unit REALISTIC is to indicate to tvhat 
extent reading, listening, and arithmetic skills may faciltate 

iob oroficiency. To contribute to the accomplishment of this objective REALISTIC ha 
Mlowed a r^odel of psychometric research frequently used m designing personnel 
selection teste, known as a concurrent validity study (Dunnette and Kirchner. 1, pp^ 
35 39) In such a study, a procedure is followed in which men who are working on a ] 
of oncirn ie tested on some skill (or personality trait, set of attitudes 
background characteristics, etc.) or area of interest and some measure of the mans 

^’"^^^ThrLlec^the^n^, is to determine how well the measures of the man’s 
predict (are related to) his job proficiency. Knowing this, it is frequen y ^ss 
improve the likelihood of selecting men who will perform satisfactorily on the job by 
aS telg the skUls test before 'a man is hired- If- his score is too low po^ 
proficiency would be predicted, and hence the man may be rejected as a candidate for 

the job. 

Defining Literacy d. 

In out application of the model described above in REALISTIC 1, wc are ekmnining 
relationships between literacy skill levels of job incumbents and various measures of their 
job proficiency. Wore dealing ‘to procedine m 

ruf“noV^\“ sir”^^^^^^ Most reading exper^ 

toirrecognlre that reading is a complex of skills and abilities. It is a so generally 
recognized rtat reading involves the ability to translate printed symbols into or 

auditory symbols, and that, in the early stages of reading training, most readers s y 
Xk7he w^ds on the printed page to themselves. As we advance in reading jk.ll there 
appears to be less need to subvocalize the words, so that there is a more direct decoding 

’’ oTUticular conceTt'us was the fact that listening ability typically precedes 
a ° r and 7fact the reading process is usually built upon a .structure of 

listening ability. Because of this we have conjectured that some of the lower aptitude 
men might no^t be able to learn a job well by reading, but would be able to ^ 

listening or show-and-tell. Their primary literacy defect might be °ne of lack ^ 
decodin'^ printed symbolism. Thus, we have decided to include listening okill 
subcomponent of “literacy.” Thus, the ability to read well presupposes and subsume 
abUitTr This idea is discussed further, below. Of secondary interes we have 

included arithmetic skUl as a measure of a specific literacy skill consisting o e a i y 
to receive and process information in a highly specific, formal, and rigorous lan^uag . 

Rationale for This Research 

The question may be raised as to why we might expect to APd^jonslups 
between literacy skills and job proficiency. Our research .s premirco on the notion that 

?) 
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language skills {including reading and listening) are primary means of learning about job 
tasks, their requirements, and how to accomplish them. Thus, in the formal haining 
programs, which precede most jobs, information (vicarious experience) is acquired in large 
part through reading and/or listening. Then, in day-by-day performance of the job, seme 
tasks are inherently reading tasks, while many others which make no immediate demands 
on reading skills may have been previously learned by reading, and hence have an indirect 
requirement for reading. Other jobs, however, may be learned by asking co-workers or 
supervisors. Such jobs, while not demanding reading skills, make a demand on listening 

skills. , , j j 

Thus, we can expect that, to the extent that job tasks make any demands on 

language skills, their successful performance will be related to some degree to measures of 
literacy skill. Yet it must be emphasized that being able to exercise literacy skills to 
obtain information about what to do and how to do it is only one (complex) factor in 
the set of factors (e.g., personality, attitudes) that determine job proficiency. We do not 
expect that knowledge of literacy skills will permit us to account for all, or even the 
greater part of, the differences in job proficiency observed in a broad spectrum of job 
incumbents. 



RESEARCH DESIGN AND PROCEDURES 

REALISTIC I is concerned with the relationships between each of the three literacy 
skill measures (reading, listening, and arithmetic) and each of the three job proficiency 
measures (job knowledge, job performance, and supervisor ratings) in each of the four 
Military Occupational Specialties (MOSs). 

The subjects were men working on the job in each MOS. These men were being 
studied under an ongoing HumRRO research program. Work Unit UTILITY (Vineberg 
and Taylor, 2) and the REALISTIC I literacy measurement requirements were appended 
to that effort.* In keeping with the design for UTILITY, subjects for REALISTIC I were 
high (AFQT > 30) and low (AFQT < 30) aptitude men who Were paired with regard to 
the amount of time they had worked on the job. However, for the purposes of the 
analyses presented in this paper, the only import of this pairing procedure is that the 
lower aptitude men are overrepresented in our sample. For this reason the figures we 
present for literacy skill levels should not be construed as parameters of the general Army 
population. 

The four Military Occupational Specialties (jobs) which we studied (cook, supply- 
man, vehicle mechanic, and tank crewman) were selected on several criteria; 

(1) They represent a range of types of job skills, 

(2) They are high density jobs to which a substantial portion of low aptitude 
men are assigned. 

(3) Except for the tank crewman, these Army jobs have close counterparts in 

the other services as well as in the civilian society. 

The nature of these jobs is largely self -apparent: Cooks prepare food in accordance 
with a master menu and detailed recipes; supply clerks requisition and issue supplies and 
maintain records, primarily clerical tasks; mechanics diagnose and repair motor vehicles, 
and the tank crewmen operate the tank and its weapons systems, a substantially 

procedural set of tasks, • • u 

These jobs differ widely in the number of job-related reading tasks involved in job 
performance, as well as in the literacy and arithmetic requirements for entry. For each. 




‘ ^ The cooperation of Dr. Robert Vineberg and Dr. Elaine Taylor and the members of their research 

team made the REALISTIC I research possible. 
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the training program contains a large amount of reading material and, for each, extensive 
manuals are available as part of the job equipment. 
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Literacy and Other Predictor Measures 

Reading-To assess reading skills, the Survey of Reading Achievement, Junior High 
Level' (grades 7-9) developed by the California Test Bureau was used. This test was 
chosen for the following reasons: (a) It provides a composite index of rea mg s s 
e.xpressed in school grade equivalents, (b) Grade placement norms span a wide range of 
abilities (grades 4.0-14.5), a necessary characteristic when testing men whose AFQT 
scores range from the 10th to the 99th centile. (c) It has a testing time of 40 minutes, 
which could be accommodated within the UTILITY testing schedule 

Arithmetic-Arithmetic skills were measured by use of the Survey of Arithmetic 
Achieve me rdTTunior High Level, California Test Bureau, a companion test to the reading 

instrument and selection for the same reasons. , , ^ ■ „toi 

Listening-Listening skills were measured by a non-standardized experimental te.,t 
constKi^t^r this project. It consists of three short descriptive passages presented orally 
on tape After each passage, 12 questions, calling for information contained m the 
passage, are read aloud t%vice, and after each question the subject is required to give hm 
single word or short phrase answer. (When the passages are regarded as reading material, 
they fall at the 6th, 7th, and 14th grade level of reading difficulty.) 

AFQT— The Armed Forces Qualification Test (AFQT) is a measure of trainabi y 
whicinnrsed as a “screening” measure for acceptance into all our armed services. It 
consists of 25 multiple-choice items in each of four areas: Verbal v^h^ch is vocabulary. 
Arithmetic, which is simple word problems; Shop Mechanics, which is matching of line . 
drawing of tools and mechanical equipment according to function; and Pattern 
or form perception. Corrected for guessing, scores from our four sections are pooled and 
converted to percentile ranks. The Pi'oject 100,000 “New Standards men m^Mioned by 
Dr. Sticht in the “Overview of Project REALISTIC” fall primarily in the next-to-bottom 

Maze Test-The Maze Test is an experimental group-administered test devised by Dr. 
Bernard RimJand of the U.S. Navy Personnel Research Activity at San Diego It consists 
of a set of printed Hampton Mazes, each with five numbered enty points and one 
interior goal area. Provided with a pencU, the subjects are required to determine, for each 
starting point, whether the goal can be reached from that starting point. 

Job Proficiency Measures 

The Job Proficiency Measures were developed by the Work Unit UTILITY staff in 
conjunction with Army content experts (3). There were three different measures of job 

nrofici6ncy as follows. w* i ^ 

Job Knowledge — The job knowledge measures were multiple choice tests 

(ranging from 75 to 96 items in length) for different job areas which sampled job 
knowledge content deemed by experienced job occupants to be important for compe- 
tently fulfilling the job. The test items were selected for relevance to job performance, 
not to school training. Since these were paper-and-pencil tests, the need for rea mg s i s 

was inh6rent in tEi6 task. • 

Job Performance - Job performance tests were constructed so as to represent 

the key tasks, in terms "of essentiality, frequency, and urgency typically performed by 
men in the jobs. For the most part they do not dkectly test reading skills; however, they 
do represent what men do in these jobs. Also, the tests were not free of verbal demands 
since the instructions were presented orally. Hence the listening component of literacy 
was inherent in the testing situation, though not necessarily involved in the job task per se. 
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The tests were administered by trained testing crews, at special test sites, 
using full normal job equipment. Cooks cooked on field ranges, supplymen worked in a 
fully equipped mock-up supply room, mechanics repaired trucks and tenks, armor 
crewmen performed in tanks, simulating only the firing of weapons. In short, the job 
performance tests were hands-on samples of job tasks performed m a given career field. 

Supervisor’s Questionnaire — Each subject was rated by his immediate super- 
visor who w^ directly instructedliy the study staff that his ratings would be used for 
research purposes only and would have no administrative effect on the subject. Ratings 
were obtained on two' rating scales: (a) the standard Army Enlisted Evaluatiori Report 
consisting of five or six point scale ratings on 14 attributes such as conduct job 
performance, cooperativeness, reliability, job knowledge, and responsibility; and (^b) an 
^perimental scale of 12 items such as, “Does he need more supervision on the job than 
most‘s” and “If you could, would you just as soon replace him? Scores of the two 
rating scales have been totaled to provide the supervisory rating measure of job profi- 
ciency in this study. 

RESULTS 

Table 1 shows the percentage distribution of our subjects by reading grade, for each 
MOS separately, and for the pooled samples. A full range of reading ability is represented 
and the medians cluster around the eighth grade level, which is considered by many 
today as the defining point of functional literacy. Despite the deliberate, systematic 
over-sampling of lower aptitude, Category IV men in this study, we find a considerably 
smaller percentage of men at the lower reading levels than would be expected m light o 
data currently reported for “New Standards,’’ lower aptitude men. This may be due m 
part to the requirements for classification in these MOSs and, perhaps, to differential 

attrition throughout the training and assignment sequence. 1 . 1 /^ 

Table 2 presents an intercorrelation matrix of a set of our predictor variables (and 
some additional ones from Work Unit UTILITY) computed from the pooled sample of all 



Table 1 

Percentage Distribution of Subjects by Reading Grade Level 



Reading Grade 



4.0 - 4.9 

5.0 - 5.9 

6.0 - 6.9 

7.0- 7.9 

8.0- 8.9 

9.0 - 9.9 

10.0- 10.9 

1 1.0 - 11.9 

12.0 - 12.9 

13.0 & Over 

Total Percent 
N 



Tank Crewman 



Mechanic 



Supplyman 



Cook 



1.5 


1.5 


0.5 


4.2 


6.0 


7.6 


2.7 


1 1.6 


13.5 


14.6 


12.9 


17.4 


19.8 


21.5 


19.4 


23.4 


16.7 


14.1 


15.2 


13.7 


19.3 


17.9 


21.7 


10.8 


1 1.7 


1 1.4 


11.9 


7.9 


2.9 


2.8 


4.5 


3.9 


5.2 


7.1 


5.5 


3.2 


3.4 


1.5 


5.7 


3.9 


100.0 


100.0 


100.0 


100.0 


384 


396 


402 


380 



Total 



1.9 

6.9 
14.6 
21.0 
14.9 
17.5 
10.8 

3.5 
5.3 

3.6 

100.0 

1562 
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Table 2 



Intercorrelation Matrix of Predictors— Pooled Sample 



1 Reading 

2 Listening 

3 Arithmetic 

4 AFOT 

5 Age 

5 Education 

7 D-48 

8 Oral Directions 

9 Maze 



X 


.54 


.69 


*".65 


--.04 


.30 


.54 


.59 


.54 


X 


.43 


.47 


.00 


.12 


.40 


.55 


. .69 


.43 


X 


.61 


-.10 


.34 


.61 


.57 


.65 


.47 


.61 


X 


.01 


.26 


.60 


.53 


-.04 


.00 


-.10 


.01 


X 


.01 


-.14 


-.16 


.30 


.12 


.34 


.26 


.01 


X 


.22 


.22 


.54 


.40 


.61 


.60 


-.14 


.22 


X 


.56 


.59 


.55 


.57 


.53 


-.16 


.22 


.56 


X 


.29 


.25 


.30 


.30 


-.07 


.06 


.41 


.35 



.29 

.25 

.30 

.30 

-.07 

.06 

.41 

.35 

X 



four MOSs. Omitted from this tabJe ;are a large number of Army Classification Battery 
scores which have a pattern of .Efilatkmships similar to that of the AFQT- The last three 
tests feted do not rely on reading. Tlhe D-48 consists essentially of number senes tasks mi 
which! the series to be completed is presented, not in conveaibonal numerals, but by 
mean^. of a series of standard dominoes. The Oral Directions test gives just oral directions, 
but they are not simple. One of the easier items: “If A comes ^ter B in 
make an X in the third square, but if B comes after A, make an X in the last circle. Tht 

Maze test has been previously described. . , ,. ... . at^ht 

Examining Table 2, one pattern of note is that reading, arithmetic, and AFQ l, all 

tests requiring reading ability, are substantially intercorrelated-which is not surprising 
since one-quarter of the AFQT is vocabulary and one-quarter arithmetic word proble^ 
These three tests show similar profiles over the other predictors. Listening, a verbal but 
not reading test, presents a somewhat attenuated version of this pattern. Other data in 

Table 2 will be referred to later. ‘ j 

Before turning to the measures of job proficiency in Table 3, it should be noted 
that we have found an extremely wide range of job experience in our subjects and 
substantial relationships between experience and job proficiency. Our samples contain 
men with from less than one month to more than 15 years job experienc^ Mean 
experience is 28 months while the median is only 13. In order to remove the effects of 
ex^rience from our proficiency measures, as well as to deal with problems of curvilin- 
earitv of regression and heteroscedasticity, all job proficiency measures were adjusted. 
This involved performing a log transformation of the time-on-the-job experience measures. 




Table 3 

Intercorrelations Among Adjusted Job Proficiency Measures 





Knowledge With 
Performance 


Knowledge With 
Supervisor Ratings 


Performance With 
Supervisor Ratings 


Tank Crewman 


.49 


,20 


.14 


Mechanic 


.49 


.17 


.15 


Supplyman 


.64 


.27 


.23 


Cook 


.47 


.28 


.19 
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and computing the regression equation for log time-on-the-job and job proficiency. 
Following this, the regression-based score predicted fiom job ex^nence alone was 
Lmputel for each man. This score was then subtracted from the raw proficiency 
measures. Our adjusted proficiency scores, then, are indmdu^ delations from that value 
of job proficiency attributable to experience and thus have been freed of the differential 

effects of iob experience, %Ar\^ 

The interrelationships among the adjusted job proficiency measures for each MOS 

are shown in Table 3. The relationships among proficiency measures are consistent from 
one job to another although the commonality of these three aspects of job proficiency is 
not large The higher relationships among job knowledge and job performance in the case 
of the Supply MOS are attributed to the greater milarity of knowledge and performance 

Figure 1 presents the relationships between reac^mg grade level and job proficiency 
measures for each of the four MOSs. The ordinate ishtDiws the- percentage of men in a 
given reading abUity category scoring above an arbitir®y. mteisr.ai level which was set as 
Approximately the upper 25% of all subjects in that ■MO.&oin thsst proficiency rneasure 

The finings on the job knowledge test for tsiffik o-ewtmaa: show that none ofi the 
low readers (grade level 4.0-6.9) reached the upper 2ia% mlT the fesople criterion level 18% 
of the middle level readers (grade level 7.0-8.0) ^^^ched life levet of proficien^^^ 
of the high reading level subjects (reading grade level MO and; above) fell in the top 
quarter of our subjects. Parallel findings are shown' Cor Shffi job fsrformance measure and 
for supervisor’s ratings. As exemplified in Figure 1 , cour results exoiisistently show that m 
all MOSs, reading is most strongly related to job nanoA^rTfedge, less closely related t 
job sample measure, and in general, barely, if at aii,. ®s-:ared supervisory ratings^ With 
minor fluctuations, this figure provides a representattawe picture of our findings with the 
other literacy variables Emd with AFQT. 

Table 4 provides a general summary of our fitudings. The tabled values are corre - 
tion coefficients measuring the relationships between job proficiency measures and 
literacy variables, with other variables included as bench marks. As indicated 
relationships of our literacy variables are regularly higher with the '^i^ten . ^ 

tests than with the job performance measures-^except for the Supply MOS m which e 
job is clerical and the performance test is written. In all jobs, the relationship v^th 
supervisor’s ratings is trivial. In our data the substitution of any of 12 other aptitude and 
achievement variables for these literacy variables would make only minor changes m this 

general pattern of relationships with the job proficiency measures. mnnifnlH 

The overall picture of our findings, then, is one of a positive correlational manifold 
among a variety of psychometric measures, among measures of different assets of job 
proficiency, and between the literacy and psychometric variables and both job knowledge 
and performance. These relationships obtain, with reassuring consistency in each of the 
four MOSs despite their different job requirements. Reading and listening, which have 
considered as a specific language or literacy skill, show a consistent modemte 
relationship with job knowledge in a variety of jobs, and a similar but somewhat o 

relationship with job performance. So, likewise, does AFQT (and » ^nj^ of general 
psychometric measures) which is commonly taken as a measure of trainability, of general 

aptitude, and of “intelligence.” 
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Percentage Sconnq in Top 25% of Job Performe 



Percentage at Each Reading Grade Le\^6| Scored in the top 25% of 
Their Job Group on Each Proficiency 



Tank Crewman 

(MOS HE) 

Job Proficiency luaex 




Mechanic 

(MOS 63C) 




Reading Grade 



50 p 
40 
30 

20 H 
10 



Supply Clerk 
(MOS 76Y) 



Cook 




4-6.9 7-8.9 

Readin g Grade 



9 + 




1 



o 

ERIC 



13 



0 



Table 4 



Correlation Between Predictors and Job Proficiency Measures 



MOS 


Predictor 


Job Proficiency Index 


Job Knowledge , 


Job Performance 


Supervisor Ratings 


Tank Crewman 


AFQT 


.55 


.37 


.07 




Reading 


.57 


.32 


.06 




Listening; 


.53 


.29 


.06 




Oral Directions 


.47 


.35 


.04 




Maze 


.29 


.21 


.10 


Mechanic 


AFQT 


.44 


.32 


.16 




Reading 


.47 


.26 


.17 ' 




Listening 


.40 


.38 


.09 




Oral Directions 


.30 


.25 


.05 




Maze 


.13 


.10 


.00 


Supplyman 


AFQT 


.36 


.37 


.09 




Reading 


.40 


.40 


.10 




Listening 


.35 


.42 


.11 




Oral Directions 


.29 


.34 


.07 




Maze 


.17 


.19 


.14 


Cook 


AFQT 


.49 


.37 


.15 




Reading 


.56 


.34 


.11 




Listening 


.39 


.28 


.07 




Oral Directions 


.44 


.32 


.13 




Maze 


.20 


.21 


.05 
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Models to Aid Memory 

To help us integrate and hold the information in Table 4 in our cognitive structure, 
we have found it useful to contemplate a simple developmental model of human ability. 
This model, schematized in Part A of Figure 2, has as its first stage and core the variable 
of simple, preliterate basic intelligence or ability to deal with the environment prior to 
and independent of language development. But man is uniquely a language-using animal 
and this core rapidly becomes contained in a spoken language ability through which the 
bulk of his congress with the world is mediated. Circumscribing this spoken language 
competency is the less uniformly developed skill of ability to deal with the world 
through written language — in our application, by reading. We see these layers as inter- 
acting but, in their developmental order of acquisition, each in turn sets some limits on 
how fast and to what level the succeeding ability can develop. 

It seems tenable to us that individ ral differences in the non-language ability to make 
fundamental discriminations and assoc utions could be or are associated with similar 
differences in the ability to comprehend subtle and complicated spoken material, and this 
in turn could be associated with differences in the ability to comprehend and profit by 
information presented by the printed word. 

The major value of this developmental “model” lies in the necessity to turn it 
inside-out when we use it as a measurement model (Figure 2, Part Bj. This places reading 
skill at the center, surrounded by listening (language) which in turn is encompassed by 
the essentially non-language intelligence factor. In order to measure reading 
comprehension (setting aside fundamental decoding skills involved in the acquisition of 
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reading skill) at the core of the measurement model, we are forced, wish it or not, to do 
so by observing responses which are mediated not only by reading skill, but, also by the 
general language skill (indexed by listening in our work) and the “general intelligence,” 
non-language factor (indexed by the Maze in this research). Ascertaining listening 
comprehension is one step simpler but inevitably is the result of an unknown combina- 
tion of the language and non-language factors. 

Such considerations have led us to regard as futile any attempts to measure pure 
reading comprehension, freed of the effects of intelligence by partialing out aptitude 
scores which are so generally measured by a test whose material must be read. We have, 

rather, ordered our variables into three categories: ^ . 

(1) Variables requiring non-language (or low language) abilit-y plus aural/oral 
language ability plus reading language ability, such as our reading measure 

and AFQT. , , , i 

(2) Variables requiring non-language ability plus aural/oral language aoility, 
such as our listening measure, the Oral Directions Test; and perhaps the 
D-48 Test. 

(3) Variables requiring little or no language ability in the mediation of the 
response. This goal has long proven elusive to psychometricians and we 
have settled on the group Maze Test as an approximate exemplar. 

Support is given this position by two sets of findings. One set (Table 5) indicates 
that variables requiring reading (reading test, AFQT, arithmetic test) are more hi^ly 
related to years of education, than are the listening variables (listening test. Oral 
Directions Test), with the Maze Test performance being least related to education. 

Table 5 



Predictor Relationships With Education 





Education 


Tank Crewman 


Mechanic 


Supplyman 


Cook 


Reading 


.30 


.33 


.29 


,32 


Oral Directions 


.18 


.26 


.27 


.23 


Listening 


.19 


.15 


.16 


.14 


Maze 


-.01 


.09 


.07 


.12 


AFQT 


.28 


.24 


.29 


.26 


Arithmetic 


,37 


.26 


.33 


.32 



Tlie second set of findings in support of our “model” is shown in Table 4. These 
data indicate that the job knowledge tests, which require reading, are indeed most closely 
related to reading and AFQT, less closely related to the listening tests, and least to the 
low-language Maze Test. This pattern is found in each MOS. However, few, if any, of the 
job performance tasks contain specific reading requirements. Here the correlations with 
the reading tests (and AFQT) shrink appreciably from those found with job knowledge. 
Similarly, the relationships between the listening variables and job performance are 
smaller than with job knowledge— but the reduction is less and we find both listening and 
reading level variables about equally associated with, or predictive of, hands-on job 
performance not involving reading. The one exception to this pattern is the Supply MOS, 
a clerical job in which performance tasks make as heavy demands as does its job 
knowledge test. Finally, the non-language Maze measure maintains its lower relatioriship 
essentially unchanged between the knowledge test requiring reading and the hands-on 
performance measure. 

18 
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Intelligence, aptitude, abUity to leam— however labeled, these seem to contain a 
significant language factor. Reading comprehensioni seems usefully viewed as being 
composed of general, language-free ability plus, or multiplied by, aural/oral language 
abUity, and finally by the specific language and/or perceptual motor skills of reading. 

Measures Qif reading comprehension, or of many otherwise identified skills that are 
measured through the use of reading, seem necessarily to incorporate general language 
abUity whether wanted or not and thus tend to be measuring a common component. 
With this in mimd. Table 6 presents multiple correlations of progressively inclusive sets of 
variables with joib knowledge and performance. Although the correlations are increased by 
the use of additional predictors, the increases are not large and would be expected to 
shrink on crossrvalidation. Thus, with our data, there seems to be an indication that, 
when an inclusiive measure of reading comprehension is used (e.g., our reading test, 
AFQT) tio predict job proficiency, little additional predictive power is to be expected by 
adding further literacy measures (e.g.. ODT‘ , Listening) or “intelligence measures ^ 
Maze) to the predictive formula. This appears to apply particularly for predicting 
proficiency on bands-on, job sample performance tests. 

Table 6 



Multiple Correlations 





Tank Crewman 


Mechanic 


Supplyman ^ 


Co 


ok 




Knowledge 


Performance 


Knowledge 


Perfornnance 


Knowledge 


Performance 


Knowledge 


Performance 


AFQT 


.55 


.37 


.44 


.32 


.36 


.37 


.49 


.37 


Reading 


.57 


.32 


.47 


.26 


.40 


.40 


.56 


.34 


Listening 
Reading 1 


.53 


.29 


.40 


.38 


.35 


,42 


.39 


■28, 


Listening ) 

Arithmetic ) 


.63 


.36 


.51 


.39 


.44 


.48 


.57 


.37 i ■ 


Reading j 


















Listening \ 

Arithmetic ( 

AFQT ) 

Reading \ 

Listening 
Arithmetic 1 


.65 


,39 


.52 


.42 


.45 


.40 


.59 


.40 


AFQT ( 

D-48 / 

Oral Directionsl 
Maze 
Race 


, .65 

' 


.42 


.52 


.42 

r 


.47 


.50 


.59 


.42 
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'Oral Directions Test (ODT). 
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READING ABILITY, READABILITY, AND READERSHIP: 
IDENTIFYING JOB-RELATED READING TASKS PERFORMED 
BY COOKS, CLERKS, AND MECHANICS 



Richard P. Kem 



Data from REALISTIC II on the relationships between personnel reading skUls and 
their me of printed materials on the job will be given in this paper^As 
Caylor’s paper, preceding this one, the low reading skills of many Project 100,000 m 
Lie led ^^power specialists to become concerned that they might be assize d to jobs 
where the reading dmnands are too great, with resulting losses in job and personnel 

effecbvenejs^^^^ a basis for these concerns, in Figure 1, for three job positions, the 
average reaing difficulty level of the job’s printed materials is compared with the a^rage 
reading achievement level of high and low aptitude personnel in that jo .j 

1 discL this figure and how we obtained the data, let me emphasize the similant^ 
between the man^wer utilization concern addressed by the military s ^oject 100,000 
and current manpower utilization concerns in civilian businesses and industry. 

ir Agure 1 the reading achievement levels for two groups of perso^el are 
identified for each job positiol and labeled as “high” and “low” aptitude men. The high 
aptitude men reflect an estimate of the average reading achievement level of 
who werrSeing accepted into the job po.sitions under standard personnel selection 
nrocedures The low aptitude personnel represent a subpopulation of nien many of 
whom would have been screened out-that is, not “hired ’’"under 

nrocedures FTom this perspective, Figure 1 may reflect a situation similar to that of l^g^ 
civilian employers wh^are interested in training and employing individu^s they have 
previously tiimed away on the basis of general aptitude or low educational back^ound 
^ Now to the origin of the data in Figure 1, and the relationships between rea^bU y 

and reading achievement levels. The three job positions studied were 
fairly large numbers of the lower aptitude men are expected to be assigned to them, and 
they also represent jobs in which the skills, knowledges, and duties involved have a hign 

areas we had™rst to identify printed nmterials actually used on the job Interviews were 
conducted with job incumbents in each of the three military occupational 
(MOSs) For the purpose of estimating the difficulty levels of the reading ^^tenak 
copies of the publications identified during these mterviews were obtained, ^opie 

Mechanic’s and Cook’s publications which were cited five ^ mo^ 

For the Supplyman’s job, where few publications were cited, a copy of the most 
frequently cfted reference was obtained, along with copies of other publications which 
supply sergeants reported to be of importance and m general use 

used to assess the reading difficulty levels of the job pubhcatipns. 
appUed to a 10% sample of the pages in each of the public^ions. , 

.^ubUcation contained 100 pages, every tenth page was ^eluded m 
pages were used which contained at least one sample of a -wor 

r|c 
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Comparison of Readability of Publications in 
Three Job Fields With Reading Abilities of 
Men in Those Jobs 
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discourse; thus, if the tenth page contained only an Ulustration, or a recipe, one of the 
adjacent pages containing a 100-word sample of discourse was evaluated. 

The average grade level of difficulty was computed for each publication, and the 
average of these averages was then computed for each MOS. The readability analyses are 
summarized in Table 1, showing the total number of publications and pages sampled in 
each MOS, the range of reading difficulty levels found over all pages, and the average 
grade level of readability of materials in the MOSs. 



Table 1 



Readability Scores for Publications in Three Army MOSs 



MOS 


Number of 
Publications 


Number of 
Pages Sampled 


Range of Flesch 
Readability 
Levels (SGE)^ 


Average 

Readability 

(SGE) 


76Y Supplyman 


1 1 


64 


8.5-16+ 


16+ 


63C Vehicle Mechanic 


8 


244 


7.0-16+ 


14.5 


94B Cook 


6 


100 


6.0-14.5 


9.0 



®SGE = School Grade Equivalent 
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The data of /Fable 1 show a wide range of difficulty levels for the materials in each 
MOS Additional data indicated that of the 11 publications studied in the Supplyman 
MOS eight had average readability scores of 16 +, with the remaining three scoring 
between grade 13 and grade 16 (14.5 on the average). Next to Supply, the Mechanic 
(MOS 63C) publications were most uniformly difficult with seven of the eight having an 
average readability score of 14.5. The Cooks in MOS 94B cited the least difficult 
material All of the six publications are recipe books designed, apparently, with the users 
in mind. As indicated below, it is in the Cooks’ MOS that the materials most closely 

match the skill levels of the personnel. . • i ■ 

In Figure 1, we have attempted to portray the problem that reading materials m 

these job. positions may pose for the high and the low aptitude groups studied by 
plotting average difficulty level of job materials alongside the average reading achievemen 
levels of the personnel. ^Fhe average reading grade level scores for the high and low 
aptitude job incumbents were obtained in the course of the work reported in the 
preceding paper by Dr. Caylor. 

At the right of Figure 1, the “New Standards’’ bar represents the median reading 
ability of 46,000 new standards men (i e., assessed under Project 100,000 with AFQT 
scores in the 10-15 range), and the “Control” bar represents the median • reading a^ity of 
non-new standards men. I’hese reading ability data are from a Department of Defense 
report (2) summarizing data concerning Project 100,000, and were obtained using a 
different test (Metropolitan Achievement Test, Intermediate Level) than used m rhe 
present research to assess the skill levels of the men in the three MOSs (S^ey of 
Reading Achievement, Junior High Level, California Test Bureau). The similari y o 
assessed reading skill levels obtained with the two different tests under widely dittenng 
circumstances suggests that the estimates of reading skills for each MOS presented m 

Figure 1 are accurate. 4 . j „ 

Making the visual comparisons between readability and reading ability suggested in 

Figure 1 we note that the difficulty level of printed materials for the supply and 
mechanic job positions exceeds the average reading achievement levels of the high 
aptitude job incumbents by about five to six grade-level points. The discrepancy is even 
greater, of course, for the low aptitude supplymen and mecha,nics, ranging around eight 
grade-level ixjints. For the cooks, however, difficulty level of the printed job materials 
and the reading achievement level of the high aptitude personnel are almost numencally 
matched. The low aptitude cooks present an average reading achievement level that tails 
only about two grade level points below the reading difficulty level of theh matena s. 

If the roadability formula provides even a rough index of the difficulty ot the 
materials, then Figure 1 suggests that both high and low aptitude personnel would 
experience considerable difficulty in reading and comprehending the supplymen s and 
mechanic’s job materials. 

In general the greater the gap between readability and reading ability, the less 
comprehensible the materials are likely to be for the man on the job (6). It is also 
reasonable to suppose that the gap between the reading abUity of the men m an MCb 
and the readability of materials may influence the extent to which men attempt ^ ^se 
job-related printed materials. Data bearing on the relationships between reading ability 
readability of materials, and readership of materials (i.e., the extent of use of pnnte 
materials by men on the job) were ailso obtained by means of the on-the-job interview. 



' It is recognized that the metric systems underlying the readability and reading ability measures are 
not the same, and, strictly speaking, the two scales should not be depicted on the same ordinate. None- 
theless, this manner ol presentation, when regarded with proper caution, dramatizes the differences 
between materials and men. 
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Readability, Reading Ability, and Readership 

The questions of concern in the analyses are; Who reads the printed materials and 
what extent do they use them? 

Before presenting these data it would be well to summarize the interview procedure 
used in obtaining this information as well as the data concerning the printed materials. 

Job-related reading materials were identified by means of a structured interview 
administered to men in the target MOSs at their job locations. Table 2 summarizes the 
main items of information obtained with the structured interview. This included personal 
data, such as name and unit. We also sought information about the kinds of job activities 
the man usually performed by asking him to describe a “typical work day. 



Table 2 

Structured On-the-Job Interview to 
Identify Job-Related Literacy Materials 

Personal Data (Name, Unit, etc.) 

Describe Typical Work Day 

Five Examples of the Use of Information Sources Other Than Printed Materials 

Five Examples of Use of Printed Materials 
Obtain the Material 
Locate the Exact Page Referred to 

Five Examples of Duties or Tasks Performed Not Involving^Use of Printed Materials 
Five Examples of the Use of Arithmetic 

Ways to Modify Printed Materials to Make Them Easier to Use 



Of primary interest for this report are the two parts of the interview designed to 
obtain information regarding the man’s use of listening and reading sources in obtaining 
job-relevant information. To identify use of listening sources, the man was asked to give 
five instances when he had asked somebody for job-related information in the last month. 
Probing was continued until the man said he couldn’t think of any other such instances. 

A similar procedure was used to identify use of printed materials or reading sources. 
The man was asked to give five examples of times during the past month when he had 
used printed materials in connection with carrying out a job. In each case he was asked 
' to describe the job he had been performing and what information he had been seeking 

when he went to the printed material. Then he was asked to get the manual and to 
locate the exact page he had used, then to show the interviewer the specific parts of the 
material he had used in obtaining the desired information. This process was repeated until 
either five instances had been described or the individual could give no more, he was not 
pressed for examples beyond five in number. 

The men in this research were selected on the basis of reading grade level scores, 
with the additional restriction that their total time in the job fall between one and 18 
months. Because of entry training and leave time, 18 months tends to be the uppermost 
limit for individuals servingV^e standard draftee’s two-year tour of duty. This was the 
group in which we were prim^iX__^terested at this time. 

In order to select potential infervie:^es on reading grade level scores, so as to ensure 
the desired number at each of the thre^eai^ng level categories, it was first necessary to 
schedule special testing sessions and administer the reading achievement test. Since it was 
Q not feasible to administer reading tests to all men at a given installation who were serving 
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• ,oiii>Kio npr«:onnel records were prescreened by selecting for 

in the target job positions, a ^^^^^1 one and 18 months of job experience and by 

reading testing only men who had between 0-20, 30-50, and 

.Zf th“cr A?|t ” t:::. we„ so^^hat arbit.., the sene^ 

mtoviLed al each reading level interval tor each ot the tluee job pos.t.ons, and 
AFQT scores and reading grade level scores of each reading level subgro p. 

Table 3 

AFQT Scores and Reading Achievement Test Scores for 
Men Interviewed in each of Three Job Positions 
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n 




AFQT 


Reading 


1 Grade 


Subgroup 


A/ 


Mean 


Standard 

Error 


Mean 


Standard 

Error 



Supplynnan 

High Reading Level (9.0+) 
Middle Reading Level (7-~8.9) 
Low Reading Level (4—6.9) 

Mechanic 

High Reading Level (9.0+) 
Middle Reading Level (7—8.9) 
Low Reading Level (4—6.9) 



Cook 



High Reading Level (9.0:E) 
Middle Reading Level (7-8.9) 
Low Reading Level (4—6.9) 



11 

9 

10 

25 
34 

26 

15 

16 
17 



73.82 

36.00 

17.41 

45.16 

33.26 

24.73 

63.60 

35.12 

24.65 



4.97 

3.24 

3.01 

4.42 

2.70 

2.49 

3.76 

3.61 

4.14 



11.19 

7.73 

6.27 

10.14 

7.86 

6.06 

11.16 

7.99 

5.44 



0.44 

0.13 

0.19 

0.19 

0.10 

0.13 

0.37 

0.17 

0.38 



as^rvey of Reading Achievement. Junior High Level. California Test Bureau. 

T, J Vv- riofn— Thp data in Figure 1 suggested that the reading difficulty level of 

Readt^rship Da^ The d b pxreed the reading ability of the 

the job materials in some instances appears f of these printed 

men. Is there evidence that low reading TinXcatere^ in describing the 

materials than do those with higher reading levels. different instances 

interview procedure, 'ndividu^ wa, 

during the preceding ““"‘h five instances had not seemed to be a 

duties. When we designed the interview proced^^^ paiticularly ditticult 

very stringent requirement, s™" ’^p„rised to find that with the exception of cooks, 

to utilize printed materials. We were surpnsed printed materials 

very few individu^s co^d reca ive d^ff indicated that they rarely used 

^'^int^r— ffnd VafthTlHrU instances they had cited were the oniy times 
they had used printed materials while on the assumed that the number of 

instances e°"d of ut'flriXd^rd:! c?^d b^ 

ir are”p= rtar^ “gr“p{T for^ the thrL reading 

achievement subgroups within each of the three jobs. 
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Comparison of Readership Indices for Men of 
Three Reading Skill Levels in 
Three Job Fields 



Reading Grade Level 
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In Figxire 2, the total numbers of printed citations given by the men in each 
reading ability category have been converted into a readership index. This index expresses 
the number of printed citations given as a percentage of the maximum number possib e i 
each man had given his full limit of five. Thus, a readership index of 20 means that the 
particular group of men produced on the average, one citation of the use of printed 
material out of a possible five requested; an index of 40 would mean they produced an 
average of two printed citations out of the five requested, and so on. ... * 

Readership indices for the supplymen and mechanics (Figure 2) suggest that the 
use of printed materials increases with increases in the reading ability of the personnel 
Simple analyses of variance (Table 4) carried out separately for each job indicated that 
the increase in use of printed material with increase in reading ability level was significant 
beyond the .025 level for the mechanics, whUe for the smaller sample of supplymen the 
F fell between the .10 and .05 levels. As visual inspection suggests, reading ability 
differences among the cooks was not related to use of printed job materials. 

For the mechanics and supplymen, whose printed materials range very high m 
terms of reading dif faulty, there is evidence of a relationship between extent of usage of 
this printed material and reading abUity level of the job incumbents. For cooks, whose 
printed «naterials range around a tenth grade equivalent for readability there were no 
differences in the use of printed -materials as a function of reading ability. Of equal 
importance is the fact that men of aU three reading ability levels among the cooks 

reported high usage of printed materials. .p .u .. 

Listening-What role does listening play in the information-seeking activities of these 
job incumbents? The listening data are presented in Figure 3 in terms of the same type 
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Table 4 



Analysis of Variance Testing Listening vs. 
Readership as a Function of Reading Ability: 
Supplymen and Mechanics 



Source 


df 


Mean Square 




Supplymen 






Between Subjects 








Reading Level (A) 


2 


2.385 


1.22 


Error (Subjects w/Groups) 


57 


1.956 




Within Subjects 








Listen vs. Read (B) 


1 


7.94 


14.18* 


A X B 


2 


5.56 


9.93* 


Error (B X Subjects w/Groups) 


57 


,56 




Mechanics 






Between Subjects 








Reading Level (A) 


2 


1.945 


<1 


Error (Subjects w/Groups) 


82 


2.352 




Within Subjects 








Listen vs. Read (B) 


1 


21.14 


12.81 


A X B 


2 


ilO.57 


6.41* 


Error (B X Subjects w/Groups) 


82 


1:65 
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^ • indicates p <1.005. 

of index as used for the readership data. Simple analysis of variance carried out 

separately for each of the job positions indicated that there was no systernatic change m 

extent of use of listening sources over the three reading ability levels within any of the 

^Cten^ng vs. Reading-Is there any meaningful pattern or relationship l^tween the 
difficulty level of printed materials and the tendency to ask rather than read. Does this 

tendency vary with the man’s readmg achievement level as well as with the reading 

diffirultv level of the materials? 

Figure 4 shows the listening and readership data plotted together on the same 
graph for each job position. As stated, considering each job position sep^ately, the trend 
Sr use of listening sources, going from low to high reading ability levels, is ^ 

horizontal line. This holds for each of the three job positions Tliere is some indicat 
that the low reading abUity supplymen and mechanics may rely more on listening than 
reading for their information, but with our fairly small numbers, these data are not 
statistically significant. There are further indications in Figure 4 that summing over the 
three reading abUity levels, supplymen may use listening to a greater extent than d 

mechanics, who in turn may use listening more than do cooks. „ 4 .. 4 . fu ^ofo 

The interesting thing about this speculation (and that is all that these data 
permit) is that if these three jobs are ranked on the basis of the difficulty levels of their 
printed materials, the supplymen rank as having the most difficult, mechanics next, arid 
cooks the least difficult materials in terms of the readability estimates reported earlier in 
Table 1 and Figure 1. If these data were to be confirmed, they would suggest that, where 
the reading materials are too difficult, the men in the job may prefer to ignore the 
reading material and to get relatively more of their information by asking others 

(listening). 
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Comparison of Listening Indices for Men of 
Three Reading Slkill Levels in 
Three Job Fieldte 
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Identification of Job-Reading Tasks 

The data reported in this paper were obtained in the pursuit of the primary goal of 
this part of our research— the identification of job-related reading tasks. We see job 
reading-tasks, if properly conceived, as providing a means of examining reading require- 
ments of jobs in new and imaginative ways, so that recommendations for redesign of job 
materials, job restructuring, or remedial training geared at job-related reading skills can be 

made. ^ 

Our usual means of assessing reading achievement or skill levels utilize reading tests 

developed for use within our school systems, with norms based on student performance 
at the various, successive grade levels. In using these scores it is generally assumed that 
the successive grade levels reflect increments in the development or acquisition of some 

complex skill we call reading. x- i 

The specific relevance of reading levels, assessed in this tvay within the educational 

system, to reading behaviors of youth or adults outside of the school system is not 
directly given. It is generally assumed that the better an individual can perform on a 
grade-norm reading test, the better he or she will be able to bundle reading requirements 
of jobs and other everyday requirements. However, when it comes to identifying some of 
these specific job or^ everyday requirements and askmg the question, “What is the 
minimal reading level requirement for successful accomplishment of this reading task? 
there seems to be no . conversion table handy to. provide the answer. It is also possible 
that the reading behaviors are aligned along such different dimensions that a useful 
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Rpraus^ of these types of considerations, one of the 
conversion ' esent research was to identify and describe job reading behaviors so 

that thVtelationships between the performance of these reading behaviors and conven- 

tidnal^ean^ ^ ^se^ing^^ ‘o'‘lhe‘’s‘tu?y and identification of job reading 

msKs^s to Wendfy L^pecl of the materials that would tend to increase the require^ 
!SntsTla«d Oh the individual to detect, process, and transform inform^ m to the 

for^ required in order to implement a given job duty or task. Information 
reldilv identified as relevant to the specific job function, and which is m a directly 

respect to that job .^unction, places r«^ on 

the reading or general information processing proficiency of the individual. 

wClilieve that by using this type of identification scheme we ^1^ ab^^^^ 
Characterize the lew-1 o? information processing requirements specific job systems place 
oli'^^Svidual and will also be in a position to determine where and how these 

1hr?en«al"faf^^^^^^^^ - are identifying job reading 

ta^irs^n f»|a joint f—n 

saSfy‘^he intormaUon'^equirement. Both information sought and content objectives are 

Classified on the basis of the same system of categories. rcadine 

C present system of categories for accomphshmg the identification of readmg 
4. 1 / in Piourp Tlus Can rightfully be called a first draft system. At this 

hi' been sucwLuUy applied in the classification of the examples of information 

29 



24 



Dffifteition of Content-Type Categories 

Ti Standards and Specifications 

Content setting forth specific rules or tolerances to which task procedures or 
the completed product must conform. 

2. Identificatio n and Physical Description 

OOWteriit attempting to symbolically represent an object via an idendfying codis 
(snock #, .nomenclature) and/or by itemizing its distinguishing physical attributes. 

3j. Proc edural Directions 

Ccintent which presents a step-by-step description of how to carry out a spectfic 
job activity. Essential elements are equipment/materials/ingredients to be used, 
andiihow. they are to be used, with presentation organized in a sequential step-wise 
fashion. 

4. P rocedurai Check Points 

Content which presents a key word or highly summarized version of what should 
be done in carrying out a task rather than how it should be done. This content 
differs from the content classified under Procedural Directions in that it assumes 
the user knows how to carry out the steps once reminded that the step exists 
and/or reminded of the decision factors which determine whether the step 
is required. 

5. Fun ctional Description 

Content Which presents an operating (cause and effect, dependency relationships) 
description of some existing physical system or%subsystem, or an existing administra- 
tive system or subsystem. 

6. Thoery 

Content which describes a system of interrelationships among factors m a generalized 
physical subsystem or system (e.g., internal combustion engines), a generalized version 
of an administrative system, or in any type of abstract conceptual system. 



Figure 5 

sought and printed materials used as given by the mechanics. Based on earlier pilot 
applications, we are reasonably confident this same system can be used in ^safymg 
examples of information and printed materials used by cooks and supply men. There will 
undoubtedly be ways to improve this system of categories, while still keepmg it relatively 
simple, before the classification of the remainder of the materials is completed. 

The gener'al notion of identifying job reading tasks using the categories m Figure 
can be expresseB by contrasting two examples of the use of printed information to 
obtain the same information. In the first example, an individual obtains the gap specifica- 
tions for a spark plug (Type of Information Sought: Standards and S^cificatioins) from 
the context of material presented in standard paragraph format and oriented to^d 
describing the functioning of the ignition system (Content Type: Functional description, , 
standard paragraph format). In contrast, consider the example of an mdividual who 
obtains the same information (Standards and Specifications) a tabular disp ^ o 

ignition system specifications (Content Type: Standards and Specifications, tabular 



formati: ®i^se two indidduals are performing job-related reading tasks which differ in 

nature I'feese two tasks quite likely require different reading skills and behaviors and 
probabhr^-ilso differ in difficulty if assessed by a performance test. 

studies of the performance of different readmg abdity groups on the reading 
tasks cmied will, it is hoped, result in knowledge directly relevant to the Problem of 
^^timating reading skill requirements for jobs. Reading 
heinr ,C 2 ^'iSEUcted to represent the job reading tasks. When these job-related reading tes 
are the relationships will be determined between performance on the job 

readimfc; teste and standardized reading tests which reflect school grade reading achieve- 

Ltims means, we hope to gain an indication of readmg skill levels adequate for 
iperfiracsi various job reading tasks. In addition, we will be able to ascertain whether 
S .te-r^ng tasks rank order along a difficulty dimension defined by the assumptions 
of taieaansi^lexity of information processing reflected m our classihcation scheme. 
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REDUCING DISCREPANCIES BETWEEN LITERACY SKILL 
LEVELS OF PERSONNEL AND LITERACY DEMANDS OF JOBS 



Thomas G. Sticht 



The research described by Dr. Kern in the preceding paper focused on the dif- 
ferences between the reading difficulty levels of the printed materials available for 
performing a job, and the reading ability levels of the men who are supposed to use these 
materials. In two of the job areas he described, the reading demands of the materials 
were considerably above the skiU levels of the men in those jobs. 

The object of this paper is to discuss two approaches aimed at closing the gap that 
results when the literacy demands of a job exceed the literacy skills of the men avaHable 
to perform the job.' One approach is to increase the skill levels of personnel by training. 
The other is to modify the literacy demands of the job by the redesign of literacy 
materials. These approaches are fairly obvious but their implementation is not simple nor 
inexpensive, and the two may have entirely different outcomes with respect to a major 
criterion for the implementation of any such action; the improvement of personnel job 
proficiency. 
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Literacy Training and Job Proficiency ^ 

One approach for closing the gap between job literacy demands and personnel 
literacy skills is to upgrade the latter through literacy training. One of the most 
frequently mentioned programs in this regard was the large scale training of functional 
illiterates in the Armed Forces during World War II. Figures reported by Goldberg (1) m 
his exteriave review of Army training of illiterates in World War II indicate that, 
subsequent to June 1, 1943, some 302,838 men received literacy training. Of these 
254,272 were graduated because of successful attainment of required standards, which 
were designated as literacy skills as possessed by the completion of the fourth graae of 

This successful performance during World War II is frequently cited by reading 
experts (Witty, 2; Robinson, 3) as an example of an approach iur upgrading the literacy 
skills of adults to secure better, more competent, job performers. However, the evaluation 
of the effects of literacy training on job proficiency w^ klmost nonexistent, and sue 
evaluation as was attempted was fraught with methodological difficulties. 

In one such attempt, made by Hagen and Thorndike (4), the records of 1,026 
illiterates who entered the Navy during 1944 and who received literacy training at Camp 
Peary were compared with those, of 1,021 normal control cases who entered at the same 
time and from the same parts of the country. The records of 999 marginal aptitude nien 
were also studied to give a comparison with men comparable to illiterates but who Jd 
not receive Uteracy training. Unfortunately, several methodological limitations restrict the 
conclusions of this study. For one thing, the control group contained no enlistees. For 
another, the mar^ai group was superior to the illiterate group in literacy, gener^ 
intellectual ability, and education level. The marginal group also differed in age ^d 
background, and in many cases records were incomplete and inconsistent. Thus, the 
UUterates who received training were not comparable to the control group of margmal 
aptitude men. 
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But at any n>te, relative tu the T^aln“eTtyT^^^ 

who received literacy ® b q misconduct, 

lower proficiency ratings, more ^ P ^ controls. Thus, in this case, literacy 

SngTd" otttrta%T„‘‘du”cin" Incomparable in iob proficiency to marginal 
aptitude men who "°rrXed ‘o 

conducrLra^y t^dnin^ and melICsIoIdIerow‘’tte s’o 

literacy training. They were ^aduated fr F (i e marginally literate men 

4.9 level. When .-‘J comp^ble „„ 

^^rflr'^nranT^I^Jfdgf tests giv^n at the end of b^ic tjmnin| DiKerermes were not 

considered of any practical significance, training on job proficiency in 

Additional studies to .f Mil, ary" Service (6J. 

the Armed Services are reviewed in the ^ ^ demonstrated to result 

These studies show that little ''n ^Tm^Zt to point out, 

from the provision of trainmg in i - j _* nroviding literacy skills in excess of 

=:.rnirh- 

Apart frdtn the -mewhat linuUd 

?Ii:rol1ob"i:™.^So the hyp^^^^^^^^ 

will improve job proficiency remains it’wUl have to extend at 

Ltt foTV-l t r^^aiTevI. pus not *o ^T^i" 
or eighth grade reading skills cannot Pf""™ “ at less Uterate 

from EEALISTIC I, presented ml‘ P"*- «>ose jobs can be, and 

men can perform a variety of jobs. B“t f°' J imitation. However, as was 

rotrlvelTn rs^lol^Ihributy^^toiratlr a^ove the seventh ^de level was 
associated with higher job proficiency. 
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Modifying Job-Related Reading Demands 

An alternative to remedial literacy is^t° ^av^L^en doin^^s^)!^^^ research to 

d^t^nf fhf IS ofTowerin^g the' reading demands of jobs, and job training 

--ZV^tuirhf rnnrt:rbiSo°;a:l"yT^^^^ 

reading for some instmctional p p • - ’ compared reading and listening 

800 articles on Of 15 studies using adult subjects (mostly 

test performance for children ^nd admts ui Ustening test scores, six 

college students), eight ^ ^ listening was best with narrative material, while 

favored listening, and one reported tnp iisieniug w«. 

reading was best with descriptive material. 
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In 18 articles annotated by Duker, children were the subjects. The general finding 
was that in the early grades (first through fourth or fifth) listening was likely to be more 
effective than reading, whUe in the higher grades reading became as effective, or more 
effective, than listening. 

These studies suggest that, in some instances, listening might prove a more effective 
training technique than reading. To examine the feasibility of this idea, research was 
initiated to determine the relative effectiveness of reading and listening as means for 
instructing men of different aptitude levels, as determined by the AFQT. The possibility 
that listening might prove superior to reading in the case of the lower aptitude men was 
of particular interest. 

Reading vs. Listening* 

To compare learning by listening with learning by reading, two comprehension tests, 
\ and B, were administered to 96 Army inductees. -Forty subjects formed a low mental 
aptitude ’group with a mean AFQT score of 18.25; 56 subjects formed an average mental 

aptitude group, with an average AFQT score of 63. 

Both tests A and B served as reading and listening tests. Both had four subtests 
having a grade difficulty level of either 6.5, 7.5, or 14.5, as determined by a modified 

Flesch formula. ■ j. 

The low mental aptitude and average mental aptitude groups were split into two 

equal subgroup®. One subgroup listened to test A and read tost B, while the other 
subgroup listened to test B and read test A. In the analysis of results, the scores for the 
counterbalanced group® were combined. Reading and listening times were equated. The 
listening test was presented by means of 'a tape recorder, with gain adjusted to a 
“comfortable” listening level. Questions for the listening test were read aloud twice. Test 
questions' demanded factual information and were of a “fill-in-the-blank” nature. The 
results of this comparison indicated that none of the differences, between reading and 
listening within group® were statistically significant (Table 1). There was an overall 
decrease in pierformance as the difficulty level of the material was increased, but there 
was no interaction of mental aptitude with difficulty of material. There were, however, 



Table 1 

Scores for Low and Average Mental Aptitude Groups on 
Listening and Reading Subtests 





Average Grade Level of Material 


Aptitude Group 


6,5 - 


7.5 


14.5 




Listen < 


Read 


Listen 


Read 


Listen 


Read 


Low Mental Aptitude (/V = 40) 
Mean 
SD 


52.87 

19.96 


^ 51.68 

" 27.39 


52.46 

16.25 


42.98 

19.48 


25.48 

13.43 


26.02 

17.83 


Average Mental Aptitude (A/ = 56) 
Mean . 

SD 


72.25 

19.93 


. j 

(72.46 
i 21.04 


69.54 

15.88 


65.18 

18.45 


44.89 

19.76 

1 


48.72 . 
20.63 






* The assistance of SP 5 Pat Ford in the conduct and analyses of data for these studies is acknow! 
edged. A detailed report of this and related work is found'^jn a HumRRO Technical Report (9). 
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large differences between the two aptitude groups on both tests. This is not surprising, 
h^v^ver, since the groups were based on AFQT scores, and as Dr^ Caylor has noted, such 
scores are derived from tests requiring considerable verbal and reading stabs. 

mese data indicated that a variety of tactual information may be communicated 
equal™ well through reading or listening to men of average and low aptitudes. 
n?aterials via listening does not necessarily . mean that the poorer reading man, found n 
the lower aptitude groups, will learn better by listening than by reading. If the is 

too difficult, they do not learn much by either modality, and if it is simple, they ar 

“i"^"“prLllttofoc'' informa may not hav. much of a 
differential effect upon learning acquisition, there may be 

tips of Ustening and reading materials. In our work, for instance, 2 % of a sample 
soL 30? men would prefer to learrt by listening, ll^us, providing listening materials 

might serve to motivate learning in a significant proportion of men. 

Rate-Controlled Recordings (Time-Compression of Speech) 

One drawback to listening for instructional purposes is that the 
limited bv the speaker’s rate of speech, whereas in reading, one may vary ’rate o 
ntormattoi tatafe to suit the nature of the reading material. However, recent develop 
™nb (renewed in Foulke and Sticht, 10) have provided techniques t^t permit the 
speed, rate of recorded passages to be speeded up or slowed down without othermse 
ciTanging the speech quality. Slowing the speech rate causes the ^ ® 

the Sessaae to increase; such speech is called timeetxpanded speech. Speeding-up the 
olavback time of a recorded message results in the presentation of the message in e^ 
Sian the original recording time; therefore, accelerated speech is sometimes referred to as 

“"''ToTZmte 'possibility of using rate-controlled speech as a ‘-ching medium for 
men inducted into the Army, studies were performed to compare 

mental aptitudes with respect to their abUity to comprehend speech presented at various 
mm, Preidous re^^^^ (Foulke and Sticht), has indicated that the average oral reading 
mfe forTwsca^termtd p readers tor the blind is around 175 wpm, with a 

sigma of some 20-26 wpm. In the present research, speech rates ran^ng from 
1 1 fhp avpratTP fi e 125 wom) to eight sigmas above the average (i.e., 375 wpm) were 
u^d'" In addition to ‘different speech rates, materials presented 

trades 5 5 8 5 and 14 5 (modified Flesch formula of Farr, Jenkins, and Paterson, )• 

^ The ^esulte ofrhri ^ are shown in Figure 1, in terms of the percentage of 

rra=f^rhttes,Then“^^^ 

' Ses*arto ta^oSnate cl'ne”rthe ‘‘a^es'The 'three 

* ’mes , • . a.rifVnrMif ibpir listening to the experimental messages. The three 

' . testing control subjects without tbeir listening to uie ^ _ .rmarPs = grade 

‘ levels of diific’jlty of the messages are represented as; circles grade 5.5, q gr 

’’5^iiysef'?f‘vSra^ce on the data Indicqted a significant triple interaction 

Of soeech rate message difriculty, and mental aptitude. Add:-tional analyses indicated that 
■tL ^teril rate interaction was significant f^r the low aptitude men, out not 

' for hTS aotitude men. Figure 1 shows that the gain in information across matenals 

bwas constant for the higher aptitude men at the diffetent ^ 

pnfifude men showed a higher gain from the easier matenals at reduced rates of speech. 

' ^ With regard to the main effects of speech rate, these data indicate that s^ech rates 

! of SsVwpm and above are distressingly fast, with comprehension returning oo baseline or 
If LXe levels of performance for both aptitude groups. It is also of interest to note 
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Listening Comprehension Test Scores for Messages on 
Three Levels of Difficulty Given at 
Different Rates of Speech 
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Low Mental Aptitude Men 
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Figure 1 

that the optimal speech rate for learning appears to be slower for the lower aptitude men 
than for the higher aptitude men. However, before such a conclusion is gener y 

acceptable, replication is needed. 

A further evaluation has been made of the comprehensibility of speech accelerat d 

above 175 wpm. In Figure 2 curves are shown that indicate how well men of high, 
medium, and low mental aptitude comprehended listening materials presented at 175, 
2'.'5 or 425 wpm. These data show that, for speech rates of 275 wpm, comprehension 
declined only slightly from that at the normal 175 wpm rate for all three groups. Since 
the message presented at 275 wpm can be presented in 64% of the time reqmred to 
present the message at the rate of 175 wpm, there is a 36% saving m time. 

'Fhe fact that there was such a small loss in comprehension with a 36% saving in 
time suggested that the total amount learned could be increased by presenting the 
message twice in compressed form in the same amount of time required to present the 
messag6 once in uncompressed form. 



Comprehension Scores for the Three Aptitude Groups 
as a Function of Message Presentation Rate 
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Fiffun; 3 gives the results of research in which a time-compressed recorded message 
was given two times in the same amount of time needed to give the uncompressed 
message once. The ordinate shows the percentage correct on a listening 
test. The lower abscissa shows the time-compression ratios used, while the upper abscissa 
shows the resultant speech rates in words per minute. In each instance, the upper 
numerals in the paired values on the scales indicated the compressed version presented 

^'"'^‘The open symbols show the results on listening comprehension of presenting the 
message twice. The filled symbols give results of listening one time to the mess^e 
presented at the speech rate (compression ratio) indicated by the upper numeral on the 

abscissa at that point. 

As indicated, white presenting the message twice in compressed form id improve 
comprehension over that obtained with a single presentation in compressed form, in no 
case -vas the double presentation more effective than a sin^e presentation of the message 
in uncompressed form. Similar results have been reported by others (12, 13, 14, 15, lb. 

In a second attempt to improve peak learning in a given unit of time, aditional, 
neui information was presented in the time saved by the compression process. I" «'js ««e 
five different conditions were presented to high and low aptitude men (Figure 4). In one 
condition, a message was presented at 178 wpm. In two conations the ™ fy™ 

in compressed speech form at rates of 278 and 378 wpm. Wese =P®ech rates m 

reduction in the time required for listening to the message. In two additional conditio , 
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Comprehension Scores for High and Low Aptitude Men 
for One ait^d Two Compressed Presentations of a 
Listening Selection 

Speech Rote (v/pnn) 

275 325 375 425 




k mp«aee was comoressod. But in these Conditions, additional, new compressed 

:^teria?ts" total listening time equal to that for the uncom- 

iressed message. These conditions are represented in Figure 4. 

In all Auditions, subjects were administered a comprehension test oyer ^ 

4 - of fVso vcifo nf S78 won) wh6n th© listening time was held constant 
„forma .on presented to Pig.ie 4, presenting the new 

test in the time saved by the hme compm. 

iion of speech, did not improve peak learning over that ^ ^is"was To for 

nformation relevant to the test, but presented m uncompressed ^ 

eoth high and low aptitude subjects, and for material presented at 278 and 378 wp, . 

Summary on Substituting Listening for Reading 

The work done on the feasibility of substituting listening for reading as a m^ns of 
reductog the literacy demands of jobs has been exploratory and quite . 

mte rntrolled reemdings. So far, this research indicates that men all along the 
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Listening Comprehension Scores for High and Low Aptitude Men 
Listening to Time Compressed— Time Limited, or 
Time Compressed-Time Extended Selections 




continuum of aptitude can leam certain verbal prose equally well by listening and by 
reading The work using rate-controlled recordings suggests that, in some C8^es, both nigh 
and low aptitude men can leam less difficult materials at rates of speech more th^an four 
sigmas above the mean rate of 175 wpm with only moderate losses in comprehension. 
While this may result in as much as 36% savings in listening time, how the time saved can 
be used to increase the general level of learning has yet to.be demonstrated. Our 
projected work includes further exarhination of this problem, as well as research to study 
the effects of speech rate upon particular information processing procedures, su.ch as 
“chunking” of input for subsequent recall. A major research activity to attempt to 
improve listening skills by training via audio-visual film loops is also plannea. Such loop.s 
might also be used as job performance aids. 

Design of Job'Performance Aids to Reduce Literacy Demands of Jobs 

A job performance aid . is any information storage device, such as a manual, 
checklist, or diagram, which is available to assist a man in doing his job. The present 




‘ The assistance of PFC Donald Enderby in the work on Job Performance Aids is acknowledged. 
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discussion will deal primarily with technical manuals as job aids, and mil fotms 
design of such manuals to make them more effective. There is considerable evidence (^, 
19 20), to indicate that properly designed job manuals can enable the less experience 
and less literate man to achieve acceptable job performance, with fewer errors a^ little 
or no loss in time, and with much less training than has been thought necessary This has 
been demonstrated for highly technical jobs in which complex machines and electronics 

systems are involved. , . , • x. x ■ u i 

The major principle underlying the success of well designed aids is that job-re a 

information, which is usually stored in a man’s head, is stored in the aid in such a 
to maximize the retrievability of the information. Thus, the long training or on-the-]ob 
experiences that are usually needed for a man to acquire the necessary job infoimation 

are reduced to the time spent in learning to use the job aids. - ^ 

It must be underscored, however, that it is not the mere storage uf information ip 
the job performance aid rather than in the man’s head which accounts for the effective- 
ness of the aid. Rather, it is the nature and manner of storage that is of crucial 
importance. For instance, a technical manual may contain needed information, but this 
information may be stored in such a way that the user is not able to work wtn i . 
Relevant data reported in the preceding paper by Dr. Kern showed that most of the 
technical manuals in the mechanic and supply clerk jobs were written at a level well 
above the reading ability of the men in the job, and that there was a correspondingly ow 
level of usage of the materials. In addition to the reading difficulty level, other factors 
such as format, and nature and sequencing of information, may reduce the effectiveness 

of a job manual. . , . j • u- u 

The key to the success or failure of a job manual or other aid is its design, which 

includes the nature and manner of pre.sentation of the information in the aid The 
foundation for the effective design of such materials is stimulus-response learning theory 
as appUed to jobs by human factors scientists. In applying the stimulus-response theorv 
to job aid design, the job is ^viewed as bemg composed of a set of tasfe in which che 
worker makes some appropriate responses to a stimulus which cues (calls for) that 
response. By performing a task analysis, the various cuing stimuli and their associated 
responses are identified. Decisions can then focus on determining which cuing stimuli are 
to be placed in the job aid, and how they will be displayed. 

The procedures for the production of job performance aids have been reviewed m 
detail (21) including analysis of man-machine systems, task description and analysis, 
allocation of functions to the job aid, determination of design features of aids, and 
production of the aids. I wUl describe one project for which technical manuals were 
redesigned to improve performance. Included in the description will be a comparison o 
the standard and redesigned manuals, and a summary of some of the production 
principles that HumRRO personnel and others have developed for producing pictorial job 

manuals. 



Manuals From Work Unit SHOCKACTION 

In the late fifties, HumRRO initiated Work Unit SHOCKACTION to develop 
training procedures for improving the i^rformance of the tank crew in combat, '^ree job 
manuals were produced in the form of picture guides for tank drivers, loaders, and 
gunners. The procedure used in developing these guides involved the determination of 
what each member of a tank crew needs to know in order to do his job. To this en . 
training literature and crew activities were studied, and experienced officers were con 
suited, lists of job requirements, covering the duties and sldlls of the hree crew 

positions were established (^)* j i ^ 

From the job task lists, job manuals, in the form of picture guides, were developed 

to cover the skills required of a given crew member. In developing the guides, an attempt 
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was made to make most statements in behavioral terms (e.g., “unscrew the turret lock”), 
and to use simple terms. Then most of the tasks covered in the gmdes ^ 

tryouts and revised until it was found that even low aptitude trainees could profitably 

^pl^e'^Tshows a page from the HumRRO picture guide developed for tank gunners. 

It has !Lu task steps Larly identified by numbers in large tyj^. Each step contains 
action verbs indicating the type of beha-dor the crewman is to perform In the manual, 
the articulated pictures and verbal context pro\nde a step-by-step procedure on the same 

^^ The HumRRO manual can be contrasted with a standard technical manual (excerpts 

shown also in Figure 5) which provides instructions for the same task giving e 

quadrant the end-for-end test. The standard manual also presente ste^by-step 

for the end-for-end test; however, these procedures are provided in the context °<^her 

procedures on the same page, and there are no helpful i»ustrations. T^e 

information in picture form occurs more than 200 pages ^fore the 

end-for-end test. U provides only an onenting fuU-view of the gunner s quadrant, 

arp no nictures for fniiding th© behavior of the user. 

The SHOCKACTION endeavor, and additional HumRRO research and developmen 
under RECON have led to several guidelines' tor designing training or job 
manuals in the torm ot pictorial aids. The most important ^deline .s to test the ,ob or 
trainine aid on the prospective user. Revisions should be made until a nonce can use the 
Ild io ^ide his performance without error or assistance and with Wtle hesimtion. 
Additional guidelines tor design of manuals, resulting also from HumRRO research (_, 

P'S 26 and 27, and a memorandum^) are: , ,, • „ 

— ’ . (if Conciseness: The information provided in the job manual should require a 

minimum of -codingrdTcoding, and manipulation of information. Hence the sirnplest 
possible vocabularies and sentence constructions should be used. Irrelevant ’ 

tockground material, and “nice to know” information should not be intersper.,ed with 
directions designed to guide the actions required in the task performance^ 

(2) Standardization: The presentation and organization of information sh^ 
be standardized acroill^i^d^thin each class of tasks. In this respect specific guidelines 
may be developed to control syntax (sentence structure), use of classes of verbs 
ZL, and so Tn. This reduces the need for creative writing on the part of the aid 

designer, and insures high redundancy levels for the user , , „ 

^ (3) Analysis of Complex Tasks Into Sub-Tasks: At times, to keep a perform- 

ance sequence simple, small portions of a complex task will have to be presented as a 

sub-task. of Information: The organization of content within any type of 

task should be based on al^sis of th7 nature and sequence of task 

For instance, procedural instructions should be presented in the sequence in ^h they 

are performed. This appUes to pictures, paragraphs, sentences, and phrases withm the 

sentences. Completeness: Each significant point in a task action sequence should be 
nresented Thi7 re quires "^eful task analysis and thorough development by user testing. 
Content of ^e job aid manual should be accurate and kept up-to-date, and provisions 
should be made for modifying the manual as new requirements arise. 
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'A Consulting Report on pictorial methods of instruction for the M-73 
.45 cal. automatic pistol, by J. Roger Ware, Elmo E. Miller, and James L 

2 A memorandum by HumRRO Division No. 2, Fort Knox, Kentucky, August 1967, giving g d 

lines for the design of technical manuals to be used in training. 
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These arp general design considerations that may enter into the productian of any 
job-aid manual. Where extensive use of photographs or other pictures is to occur, there 
are several additional recommendations: 

(1) Page layout; Ideally, a sequence of photographs, or drawings, illustrating 
one step, or element,' iiTa task (such as setting the micrometer scale in Figure 5) should 
be presented on one page or opposing pages so the user can see all of that step withou 

turning pag^^ Relating pictures to words : Figures, photographs, and drawings should be 

closely conelated, in proximity and relevance, to the text material. 

(3) Subjective viewpoint: Photographs and figures should be taken from the 

user’s point of regard. , , » • 4. 

(4) Con sistency of camera an gle: Do not change the camera angle (user s point 

of view) unless critical cues can be seen only from another angle. In such cases, indicate 

graphically the changes of view. . i, 

(5) View “before” the action: Photographs or drawings should show tbe 

present state of the system rather than its state after a response is made. The system 
status “after” the action should be represented by animation devices, such as arrows or 
dotted lines, corresponding to the operator’s mental image of the deiired change. 

(6) Avoid two pictures for one movement : In presenting a single movement, do 
not use two separate photographs or drawings to represent the system “before” and 
“after.” This 'requires the user (especially the novice) tq make an extremely diitic 
discrimination between pictures, even when the difference seems apparent to an exper- 

ienced person. - i.- u 

(7) Critical cues: Direct attention to critical cues by relevant animation such as 

arrows, color coding, outlining, and so forth. . » . , • . • i 

As these guidelines indicate, the development of an effective job-aid is not a simple, 
“quickie” process. Hov/ever, in the interim between the decision to develop adequate 30b 
aids and their actual delivery, a supervisor or manager may take several steps to nrnke 
existing aids more usable. Some suggested measures are; (a) provide indexes to existing 
manuals in terms of local jargon; (b) excerpt (photocopy, etc.) certain frequently-used 
segments of manuals; (c) rewrite parts of present aids in local jargon and m simple 
language suitable to even the poorest readers; and (d) encourage and provide on-the-job 
training in the location and use of existing manuals and other job aids where available. 



Summary 

I believe that most of the evidence available today indicates that the provision of 
more useable job materials may be the most rapid, least expensive method of assunng 
that job proficiency is attained by men across a wide spectrum of literacy or verb^ 
aptitude skill levels. There is even some evidence (^) to suggest that the use of printed 
materials may be facilitated more by the design characteristics of the matenals than by 
increasing the reading skills of men, and this may be true for able as well as for poor 

F0£LCl6rS 

The fact that it is possible to design materials to permit mai’ginally literate men to 
perform jobs satisfactorUy indicates that it is feasible to hire the less literate man and to 
utilize him effectively, whUe providing literacy training to up-grade these slu Is. By-this 
means, it should be possible to employ the marginally literate in responsible, growth- 
oriented jobs, while maintaining the efficiency and ef;.ectiveness of the job sub-system. 
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